274 research outputs found
Contrastive Hierarchical Discourse Graph for Scientific Document Summarization
The extended structural context has made scientific paper summarization a
challenging task. This paper proposes CHANGES, a contrastive hierarchical graph
neural network for extractive scientific paper summarization. CHANGES
represents a scientific paper with a hierarchical discourse graph and learns
effective sentence representations with dedicated designed hierarchical graph
information aggregation. We also propose a graph contrastive learning module to
learn global theme-aware sentence representations. Extensive experiments on the
PubMed and arXiv benchmark datasets prove the effectiveness of CHANGES and the
importance of capturing hierarchical structure information in modeling
scientific papers.Comment: CODI at ACL 202
HEGEL: Hypergraph Transformer for Long Document Summarization
Extractive summarization for long documents is challenging due to the
extended structured input context. The long-distance sentence dependency
hinders cross-sentence relations modeling, the critical step of extractive
summarization. This paper proposes HEGEL, a hypergraph neural network for long
document summarization by capturing high-order cross-sentence relations. HEGEL
updates and learns effective sentence representations with hypergraph
transformer layers and fuses different types of sentence dependencies,
including latent topics, keywords coreference, and section structure. We
validate HEGEL by conducting extensive experiments on two benchmark datasets,
and experimental results demonstrate the effectiveness and efficiency of HEGEL.Comment: EMNLP 202
SummIt: Iterative Text Summarization via ChatGPT
Existing text summarization systems have made significant progress in recent
years but typically generates summaries in a single step. The one-shot
summarization setting is sometimes inadequate, however, as the generated
summary may contain hallucinations or overlook important details related to the
reader's interests. In this paper, we address this limitation by proposing
SummIt, an iterative text summarization framework based on large language
models like ChatGPT. Our framework enables the model to refine the generated
summary iteratively through self-evaluation and feedback, closely resembling
the iterative process humans undertake when drafting and revising summaries. We
also explore using in-context learning to guide the rationale generation and
summary refinement. Furthermore, we explore the potential benefits of
integrating knowledge and topic extractors into the framework to enhance
summary faithfulness and controllability. We evaluate the performance of our
framework on three benchmark summarization datasets through empirical and
qualitative analyses. We also conduct a human evaluation to validate the
effectiveness of the model's refinements and find a potential issue of
over-correction. Our code is available at
\url{https://github.com/hpzhang94/summ_it}.Comment: work in progres
A Revisit of Fake News Dataset with Augmented Fact-checking by ChatGPT
The proliferation of fake news has emerged as a critical issue in recent
years, requiring significant efforts to detect it. However, the existing fake
news detection datasets are sourced from human journalists, which are likely to
have inherent bias limitations due to the highly subjective nature of this
task. In this paper, we revisit the existing fake news dataset verified by
human journalists with augmented fact-checking by large language models
(ChatGPT), and we name the augmented fake news dataset ChatGPT-FC. We
quantitatively analyze the distinctions and resemblances between human
journalists and LLM in assessing news subject credibility, news creator
credibility, time-sensitive, and political framing. Our findings highlight
LLM's potential to serve as a preliminary screening method, offering a
promising avenue to mitigate the inherent biases of human journalists and
enhance fake news detection
Extractive Summarization via ChatGPT for Faithful Summary Generation
Extractive summarization is a crucial task in natural language processing
that aims to condense long documents into shorter versions by directly
extracting sentences. The recent introduction of ChatGPT has attracted
significant interest in the NLP community due to its remarkable performance on
a wide range of downstream tasks. However, concerns regarding factuality and
faithfulness have hindered its practical applications for summarization
systems. This paper first presents a thorough evaluation of ChatGPT's
performance on extractive summarization and compares it with traditional
fine-tuning methods on various benchmark datasets. Our experimental analysis
reveals that ChatGPT's extractive summarization performance is still inferior
to existing supervised systems in terms of ROUGE scores. In addition, we
explore the effectiveness of in-context learning and chain-of-thought reasoning
for enhancing its performance. Furthermore, we find that applying an
extract-then-generate pipeline with ChatGPT yields significant performance
improvements over abstractive baselines in terms of summary faithfulness. These
observations highlight potential directions for enhancing ChatGPT's
capabilities for faithful text summarization tasks using two-stage approaches.Comment: Work in progres
- …